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This paper consists of reviews of current trends in World 



Wide Web-based instruction and a representative sample of Web-based 
instruction efficacy research. The review of the current state of Web-based 
instruction yields two principle themes: (1) the prevalence of Web-based 

instruction is growing exponentially, particularly in lifelong learning and 
corporate training; and (2) the line between academia and private industry is 
blurring at an unprecedented rate that is strongly associated with the growth 
of Web-based instruction and training. A review of Web-based instruction 
efficacy indicates that, despite the prevalence of Web-based instruction, 
there are few efficacy studies in which Web-based and face-to-face 
instruction are compared. The paper describes views of educational 
researchers who object to the utility of such studies on methodological and 
theoretical grounds. Rationale for the need for such studies is also 
presented. Four representative efficacy studies are summarized, and their 
results, taken together, yield three principal conclusions: (1) Web-based 

instruction can be as effective as face-to-face instruction based on 
students' grades and exam scores; (2) learning variables play an important 
role in terms of the nature of students who select and who succeed in 
Web-based classes; and (3) factors that lead to improved performance in 
face-to-face classes, such as structured collaboration and active learning, 
also increase performance and positive attitude in Web-based courses. 
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Abstract 

This paper consists of a review of current trends in web-based instruction and a review of 
a representative sample of web-based instruction efficacy research. The review of the 
current state of web-based instruction yields two principle themes: a) The prevalence of 
web-based instruction is growing exponentially, particularly in lifelong learning and 
corporate training; and b) The line between academia and private industry is blurring at 
an unprecedented rate, and the trend is strongly associated with the growth of web-based 
instruction and training. A review of research on web-based instruction efficacy 
indicates that, despite the prevalence of web-based instruction, there are few efficacy 
studies in which web-based and face-to-face instruction are compared. The paper 
describes views of some educational researchers who object to the utility of such studies 
on methodological and theoretical grounds. Rationale for the need for such studies is 
also presented. Four representative efficacy studies are summarized and their results, 
taken together, yield three principal conclusions, a) Web-based instruction can be as 
effective as face-to-face instruction based on students grades and exam scores; b) Learner 
variables play an important role in terms of the nature of students who select and the 
nature of students who succeed in web-based class; and c) Factors that lead to improved 
performance in face to face classes, such as structured collaboration and active learning 
also increase performance and positive attitude in web based courses. 
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The Horse and the Cart in Web-Based Instruction: 

Prevalence and Efficacy 

Prevalence 

In one of the first, if not the first article to appear in a main stream academic 
education journal on the World Wide Web and education, Ron Owston led with what has 
become one of the most commonly quoted statements in articles covering this topic: 

“ Nothing before has captured the imagination and interests of educators 

simultaneously around the globe more than the World Wide Web ” 

(Owston, 1997, p. 27) 

Ron Owston probably couldn’t have guessed how much truer this statement 
would be by the turn of the century, just a few years later. 

Virtually every major University now offers at least some on-line courses, and the 
web is also used extensively in “traditional” face to face classes. Further, a third of U.S. 
colleges now offer degrees online (Huffstutter & Fields, 2000). There are entire 
Universities that are primarily on-line, and, even traditional “brick and mortar” schools 
that are using the web as the primary delivery mechanism rather than the classroom and 
teacher approach. At Daniel Jenkins Academy in Florida, students will meet in a regular 
building according to a regular schedule, but will take all their classes on line (Thomas, 
2000). Regular textbooks are being replaced by electronic texts on some campuses. For 
example, seven dental schools, including New York University and the University of 
Texas are planning to replace four years of medical text books with one DVD for 
incoming students (Guernsey, 2000). 

A fundamental and growing trend that is strongly associated with the growth of 
the World Wide Web and instruction is the blurring of the lines between traditional 
academia and the business sector. A number of factors account for this trend, primary 
among these are the growing need for professional post-graduate training in today’s high 
tech and quickly changing work world, combined with the low cost, convenience and 
efficiency of training provided via the Web. This is the principle reason that corporate 
training via the web grew almost five fold, from .19 billion in 1997 to 1 billion in 1999. 
Further, some project that this will grow to a whopping 1 1.4 billion by 2003 (Sambataro, 
2000). John Chambers, the CEO of Cisco systems refers to “e-leaming” as the second 
wave in the Internet after e-commerce. According to Chambers “The next big killer 
application for the Internet is going to be education. Education over the Internet is going 
to be so big it is going to make e-mail usage look like a rounding error” (Mccright, 1999). 

In addition to growth within web-based training in the corporate world, more and 
more private enterprises are working their way educational arenas that have traditionally 
been the purview of traditional academia. For example, large web-based “learning hubs” 
such as Smart Planet (http://www.smartplanet.com) and Hungry Minds 
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(http://www.hungryminds.com) offer education of all sorts in areas as diverse as “how to 
find airline tickets” to links to University degree programs in computer science. Many 
of the courses are available for free because the sites rely on free education and a 
marketing technique. The company, notHarvard.com introduced the term 
“educommerce” to describe this business model (http://www.notharvard.com). Of 
course, such trends are diametrically opposed to the traditional notion that the academic 
world can and should maintain a fundamental independence from the corporate world. 
Whether academia is ready for it or not this model may be changing, and may be 
changing relatively quickly. In a recent article in the New York Times, Arthur Levine, 
the president of Columbia University, made the following troubling assessment of the 
current state of education, academia, and the business world. 

You know, you're in an industry, which is worth hundreds of billions of 
dollars, and you have a reputation for low productivity, high cost, bad 
management and no use of technology. You're going to be the next health 
care: a poorly managed nonprofit industry, which was overtaken by the 
profit-making sector ... Those of us in higher education have a small 
amount of time to stop and think. What is the purpose of higher education? 

How shall we continue to accomplish it? Not to answer these questions is 
to make a profound decision, by default, about our own prospects for the 
future. (Levine, 2000) 
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The Case against Efficacy Studies 

Amidst the dramatic changes in the nature and philosophy of higher education 
associated with the growth of the web in instruction, a central question remains. Are 
web-based learning techniques as effective as traditional face to face instruction? Of 
course, given that both face-to-face instruction and web-based instruction can vary 
widely in the nature of their instructional design, the more answerable question is, “Can 
web-based instruction be as effective as face to face?” or even more answerable, “Is web- 
based instruction in its current form as effective as face to face instruction, within the 
context of some subject domain.” 

Although such a question would seem on the surface to be an obvious and 
important first step in web-based learning, surprisingly few such studies have been 
conducted. One reason for this lack of research is probably due to the fact that some 
educational researchers deem such a comparison to be impossible due to inherent 
methodological and theoretical flaws. Richard E. Clark (1983; 1994) and others contend 
that it is impossible to separate the instructional media from the instructional technique, 
so that, for example, any comparison of web-based instruction to face-to-face instruction 
is actually a comparison of different instructional techniques and strategies that are 
brought about as a result of the media (Russell, 1999). As support for this contention 
they point to the bulk of instructional media studies that find no significant difference 
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between the use of different types of media. Further, others have complained that there is 
an inherent instructor bias in instructional media studies. First of all, media type is often 
confounded with instructor in such research. For example, if one instructor creates or 
moderates a web-based class and another instructor teaches the face to face class in a 
comparison study, it is impossible to tell if outcome differences are due to instructor or 
media. Further, if the same instructor is used for both conditions, there is often a bias in 
that the instructor is the one who has invested time and effort in creating the media, or, is 
in some other way biased in favor of the media group. This creates a bias in terms of 
motivation and expectancy on the part of the instructor. Finally, it is also often 
impossible to randomly assign students to web-based versus face-to-face classes for 
pragmatic and/or ethical reasons, and self-selection biases occur. In fact, the survey of 
studies below indicates that there are substantial differences between those who select 
web-based classes as compared to face to face. 

The Case for Efficacy Studies 

Despite these objections there remain a number of arguments in favor of carrying 
out efficacy studies in some form. First, given the trends cited above, and the large 
amount of resources invested in web-based education, many in the society who are 
effected by web-based instruction (e.g., parents, educators, government leaders) are 
calling for evaluation of the use of the web in education. Despite methodological flaws, 
many educational researchers would argue that we owe it to the society at large to do our 
best to evaluate these techniques carefully, and to share our findings, while emphasizing 
important qualifications in interpretation. 

There have always been methodological limitations within applied education 
research studies, particularly when a new instructional technique is examined, whether it 
involves instructional media or not. However, many researchers still see the need for 
ecologically valid studies conducted in applied contexts, and view the methodological 
limitations as challenges that can be addressed in many creative ways. This same 
philosophy can be applied to efficacy studies involving web-based education. One 
strategy is simply to present the findings within the context of these limitations, so that 
the researcher emphasizes that the results must be considered with given qualifications. 
Further, applied research that is lacking in the systematic controls and rigor of more basic 
educational and cognitive research can serve as one component of a larger research 
design, which includes more basic research. Combining applied and basic research 
aimed at examining such phenomena as web-based instruction can serve as a very 
powerful tool for discovering important factors in accounting for the efficacy of such 
techniques. Finally there are many creative alternative research tools that the applied 
research can use in studies of web-based instruction such as the measurement of multiple 
outcomes and leaner variables, and subjective-quantitative and qualitative techniques. 

Representative Studies Description 
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Introduction. Below I will review and summarize four web-based vs. face-to-face 
comparison experiments. This is not an exhaustive review by any means; however, the 
number of such studies is surprisingly small. The first experiment was selected because 
it received a lot of publicity within popular education literature, and the second study was 
selected for inclusion because it was conducted specifically in an effort to replicate the 
first. The third experiment was selected due to the focus on learner variables, and the 
fourth was sleeted since it is almost surely the largest, comprehensive study conducted up 
until the present. 

Schutte (1996). One of the first comparison studies of web-based and traditional 
face to face instruction was carried out by Jerald G. Schutte in 1996 in a Sociology 
Statistics class. His results received a lot of attention in educational circles and the 
popular press including an article in the Chronicle of Higher Education (1997). Schutte 
randomly divided students into a web-based vs. face-to face class and found that students 
in the Web-based class performed dramatically better. They scored 20% higher on both 
the mid term and final tests. Those in the Web class were also significantly more positive 
on self-report measures of math attitude, and also, surprisingly, reported significantly 
more contact with class members than those in the face to face class. Those in the web- 
based class also reported spending significantly more time on the class. 

Unfortunately most of the press about this experiment did not point out that some 
aspects of this study almost certainly preclude the conclusion that the web-based 
instruction is significantly more effective than face to face, even within the context of this 
specific class. As others have pointed out (Neal, 1998), there were substantial, not so 
subtle, differences between the web and face to face classes independent of the World 
Wide Web. Those in the face to face class only met once a week on a Saturday for 
several hours, and spent the whole time working math problems. Those in the web class 
also worked these same problems and submitted them via web forms. In addition, those 
in the web class did email assignments in 3 person groups that included generating 
weekly reports, responded to a weekly discussion topic twice a week, and participated in 
a web-based chat session with the instructor once a week. The problems in experimental 
design in this study are obvious, nevertheless, I do believe the experiment adds useful 
information to the body of knowledge on web-based instruction. Among other things, the 
experiment demonstrates that fundamental principles of effective instruction within a 
face-to-face class, such as encouraging active learning and collaboration on the part of 
learners extends to web-based instruction as well. 

Jones (1999). In an effort to replicate Schutte’s experiment, Edward Jones carried 
out a similar Web vs. face-to-face comparison within an undergraduate statistics class. 
One fundamental difference between the experiments was that students in the Jones 
experiment were not randomly assigned to groups. The format for the web class was 
similar to Schutte’s in that it included on-line lecture notes, quizzes, a discussion 
listserver, email, and biweekly chat sessions. Unfortunately, there is not much detail 
listed in the research report with respect to the face to face class, or whether or not the 
students in the web class were simply required to carry out more activities and 
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assignments than the face to face class, such as was clearly the case in the Schutte 
experiment. 

Due to the fact that the Jones experiment did not use random assignment, 
demographic variables were considered. In fact, some of the most interesting findings of 
this experiment involved the demographics in providing information about students who 
selected web-based class versus face to face. Jones found that those who selected the 
web course were significantly older and also had significantly higher GPAs. In terms of 
outcomes, those in the Web-Based groups scored significantly higher on exams, when 
GPA was not taken into account. However, when GPA was considered as a covariate, 
differences between the groups on exam scores were not significantly different, and, in 
fact, the adjusted mean for those in the face to face class on the final was slightly higher 
than for those in the Web class. 

White (1999). This experiment consisted of a comparison of a WWW class with a 
face to face class in Communication, Technology, and Change. As with the Jones 
experiment, students were allowed to select their instructional modality (web vs. face to 
face), and the researchers considered demographic variables in an effort to control for 
these selection effects. The web-based lessons included text lectures with images, links 
to related sites, and discussion questions posted to a listserve. Unfortunately, there is no 
description of the face-to-face class in the research report beyond the fact that they did 
not have access to the web materials. 

Those who selected the web based classes scored significantly higher on pretests 
of reported computer usage, and significantly lower on reported computer apprehension 
and communication apprehension. However, unlike the Jones study, the groups did not 
significantly differ in GPA. Interestingly, the significant difference on computer 
apprehension between those who selected the Web-Based versus face-to-face class 
disappeared on the post-test. Computer apprehension for those in the face to face class 
decreased from pre to post test and increased for those in the web-based class! The 
groups did not significantly differ on mid term and final exam scores. 

Wideman & Owston (1999) (also reported in Owston, 2000). This study 
constituted what is probably the largest and most comprehensive comparison of web- 
based versus traditional instruction to date. The study included 14 different classes, each 
of which was offered in a web-based format and in a corresponding face to face or 
traditional correspondence (paper) format. The study included 1 099 students in web- 
based courses, 2467 students in correspondence courses, and 2318 students in face to face 
classes. The web courses used Lotus Learning Space and typically weekly lessons were 
posted, a study schedule, weekly readings, lectures, and occasionally graphics. The 
courses included electronic discussion rooms but the authors note that the use of the 
rooms varied widely across web classes. 

An initial analysis of course grades found that students in the correspondence 
courses scored significantly lower than the other two modalities, which did not 
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significantly differ. Further examination of the data indicated a large number of zero 
scores in the web-based classes. Though no official records were retained, the 
researchers discovered that most of these zeros were assigned to students who dropped 
out of classes. A second analysis was conducted, which did not include stidemts wotj 
grades pf zero. In this analysis grades were significantly higher for the web-based classes 
than for the face-to-face classes, which were significantly higher than for the 
correspondence classes. 

Conclusions Based on Representative Studies 

The results of the studies summarized above, taken together, yield the following 
general conclusions. 

• Students in web-based courses perform as well as those in traditional classes, based 
on performance on tests and class grades. 

• Learner variables play an important role in selection of, and performance in, web- 
based courses. 

• those who select web courses are more computer savvy, older, and are, in some 
cases, better students (i.e., higher GPA) 

• Students in web-based courses are more likely to drop out; indicating that self- 
motivation may be a more important factor in performance in web-based classes 
than in traditional classes. 

• Factors that lead to improved performance in face to face classes, such as structured 
collaboration and active learning also increase performance and positive attitude in 
web based courses. 
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